fix(brightdata): fix async Discover API, echo-back fields, and registry ordering#4188
fix(brightdata): fix async Discover API, echo-back fields, and registry ordering#4188waleedlatif1 merged 6 commits intostagingfrom
Conversation
transformResponse receives params as its second argument. Use it to return the original url, query, snapshotId, and searchEngine values instead of hardcoding null or extracting from response data that may not contain them. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Bright Data Discover API is asynchronous — POST /discover returns a task_id, and results must be polled via GET /discover?task_id=... The previous implementation incorrectly treated it as synchronous, always returning empty results. Uses postProcess (matching Firecrawl crawl pattern) to poll every 3s with a 120s timeout until status is "done". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move box before brandfetch/brightdata to maintain alphabetical ordering. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
PR SummaryMedium Risk Overview Updates multiple Bright Data tools’ Minor cleanup: reorders Reviewed by Cursor Bugbot for commit 9e9bf35. Configure here. |
Greptile SummaryFixes the BrightData Discover tool to use the async POST→poll-GET pattern (the API returns a Confidence Score: 5/5Safe to merge — all remaining findings are P2 style suggestions that don't block correct behavior. The two prior P1 issues are resolved, the polling implementation follows the established Firecrawl pattern, and the echo-back fixes are clean. Remaining comments (poll error body, content fallbacks) are speculative improvements that don't affect the core correctness of the fix. apps/sim/tools/brightdata/discover.ts — poll error message and content field fallbacks worth a second look before merging. Important Files Changed
Sequence DiagramsequenceDiagram
participant W as Workflow Executor
participant T as brightDataDiscoverTool
participant BD as BrightData API
W->>T: execute(params)
T->>BD: POST /discover (query, intent, ...)
BD-->>T: { task_id: "abc123" }
T->>T: transformResponse → output { taskId, results: [] }
Note over T: postProcess polling loop
loop every 3s until MAX_POLL_TIME_MS
T->>BD: GET /discover?task_id=abc123
BD-->>T: { status: "pending" }
end
T->>BD: GET /discover?task_id=abc123
BD-->>T: { status: "done", results: [...] }
T->>T: map results → { url, title, description, relevanceScore, content }
T-->>W: { success: true, output: { results, query, totalResults } }
Reviews (3): Last reviewed commit: "fix(brightdata): use platform execution ..." | Re-trigger Greptile |
The executor wraps postProcess in try-catch and falls back to the
intermediate transformResponse result on error, which has success: true
with empty results. Throwing errors would silently return empty results.
Match Firecrawl's pattern: return { ...result, success: false, error }
instead of throwing. Also add taskId to BrightDataDiscoverResponse type
to eliminate unsafe casts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@greptile |
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit b549ee4. Configure here.
Replace hardcoded 120s timeout with DEFAULT_EXECUTION_TIMEOUT_MS to match Firecrawl and other async polling tools. Respects platform- configured limits (300s free, 3000s paid). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@greptile |
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 9e9bf35. Configure here.
Summary
task_id, poll GET for results) usingpostProcesspolling, matching Firecrawl's crawl patternurl,query,snapshotId,searchEngine) across 5 tools to useparamsfromtransformResponsesecond argument instead of hardcodednullor unreliable response extractionboxbeforebrandfetch/brightdata)Test plan